Due to the continuous growth of Internet traffic, methods of performing real-time, non-invasive flow measurements are becoming more and more important for operators to monitor network performance, detect router availability, troubleshoot network congestion, and measure QoS without introducing additional traffic. This is especially true for service providers who have to guarantee the Service Level Agreements (SLAs) to their customers.
Today, there are several solutions or mechanisms both active and passive, to tackle the underlying problem. Some of these solutions are set out below.
In today's networks, switches/routers individually store and collect a set of traffic statistics into their built-in Management Information Base (MIB) thus providing a view of the traffic performance limited to a node with no comprehensive view of the traffic behavior at the network level. A network or element manager can retrieve these traffic statistics by using the SNMP management interface of the routers. Typical traffic statistics could be the number of discarded packets, error packets, port utilization, buffer utilization, etc. These statistics are collected for later analysis. When a network congestion or event occurs, the SNMP agent embedded in the IP router will send a trap message to the SNMP manager, which then indicates an alarm in its graphical user interface.
The network management system for IP routers usually involves the SNMP manager and SNMP agent, which use the SNMP MIB for storing control information and performance statistics. However, the MIB is embedded in the IP router and corresponds to the structure and implementation of the underlying hardware. Once the design is completed, changing the MIB to accommodate new traffic performance parameters of interest is not easy.
The router MIB is designed based on the specific structure and implementation of the IP router, and therefore will not be the same for equipment from different vendors. For example, the Argent Guardian tool from Argent Software Inc. has different versions for performance monitoring, proactive problem detection and correction depending on the monitored entities. The Argent Guardian for Cisco can only be used for Cisco routers because it uses the Cisco router MIB to retrieve and query the traffic information.
Active performance monitoring mechanisms send special test-packets to the underlying networks or routers, and analyze the response. Currently, most tools for monitoring QoS in IP networks are based on the traditional “ping” (i.e., ICMP and echo response messages) to measure the roundtrip delay between two hosts. Variations of ping include “Nikhef ping” and “fping”. Several other tools are based on the traditional “traceroute” such as “Nikhef traceroute” and “network probe daemon”.
The PingER project at SLAC (Stanford Linear Accelerator Center) uses repeated pings between ESnet (Energy Sciences Network) sites. The AMP (Active Measurement Program) project by NLANR (National Laboratory for Applied Network Research) performs pings and traceroutes between NSF-approved high-performance connection sites. The NIMI (National Internet Measurement Infrastructure) project measures the performance between sites using traceroute or TCP bulk transfer.
The typical passive traffic analysis is performed in a non-invasive way with respect to the observed networking environment. It will not introduce any additional traffic and therefore will not affect the performance of the network while doing measurements. The traditional approach usually includes:                collection of TCP/IP packets (e.g. traffic sniffers, etc.) or the packet header data,        hardware and software for analyzing the collected databases and finally,        off-line traffic characterization and modeling.        
The National Laboratory for Applied Network Research (NLANR) has been using the OCXmon monitors to tap into the light of a fiber interconnection by means of optical splitters, and collect packet header traces. Traffic data was collected in an abstract format by extracting and storing the packet header in the database within a preset traffic aggregation period. It is followed by an off-line traffic analysis. System vendors all have proprietary solutions to collect statistics more at the flow level. Cisco offers a NetFlow capability in its large routers. NetFlow is able to identify traffic flows based on IP source/destination addresses, protocol ID field, type of service (TOS) field, and router port. Statistics can be collected for a traffic flow, and exported to a collector when the flow expires. Flow statistics may include the flow start/stop times, number of bytes/packets, and all IP header fields.
An IETF proposal (RTFM, RFC 2722) also targets the provisioning of flow monitoring capability, even if not yet present in any industry product. Chip vendors often propose statistic built-in solutions (especially TCAM vendors, with a number of entry hits for instance). Lucent Bell Labs has various research projects in traffic analysis, which focus mainly on collection of TCP/UDP/IP packet header data, off-line traffic analysis, modeling and visualization.
Thanks to the traditional passive analysis mechanisms, many traffic studies have attempted to understand the random behavior or composition of Internet traffic. However, they all focus on off-line analysis of historical data. There are no prominent research projects attempting traffic analysis and control, based on real-time traffic measurement or comprehensive traffic profiling. For example, Lucent's projects reflect the traditional approach of collecting large traffic measurement data sets and off-line statistical analysis. Cisco NetFlow essentially measures the volume and duration of each traffic flow for accounting purpose and off-line traffic analysis. But NetFlow is not intended to be used for real-time network monitoring and querying. The OCXmon tool from NLANR is only for IP over ATM traffic, and is not for traffic monitoring and control purpose. Moreover, each OCXmon monitor captures just the first ATM cell from each IP packet thus providing incomplete IP trace.
But all of the above solutions are node centric, and none really adopts a network view to addresses the problem, even if a need is indirectly acknowledged throughout the interest shown and efforts spent on flow aggregation features by all interested parties.
The present invention relates to two aspects namely aggregation and correlation of packet filtering information. The aggregation task will be described first while the correlation task is described later. The invention allows distributing these tasks among all the edge routers of the network thus avoiding off line analysis by the service manager to compile their results and thus optimizing the processing of these tasks. The delay computation is presented herein as an example of a possible correlation. It will be apparent to one skilled in the art that the correlation aspect can be applied to other applications. Further more, the invention is not limited to IP networks but could be applied to any communication network.
The basic idea of this invention is to combine the advantages of two technologies, Real-Time Flow Management (RTFM) and virtual router network, to provide a flow monitoring concept at the network level, offering characterization of through-traffic in a passive way.
Today, IP routers and switches collect limited traffic statistics for network performance over time intervals. Recognizing the need for more timely and sophisticated traffic measurements, the IETF RTFM (RFC 2722) working group developed a general framework for measuring properties of traffic flows in real-time. After defining a traffic flow as a portion of traffic, delimited by a starting and ending time, IETF RFC 2722 identifies flows in terms of their attribute values such as source/destination addresses, cumulative count of bytes and packets, service type, packet size, flow state information, QoS parameters. The general framework for real-time flow monitoring is shown in FIG. 1.
The RTFM architecture includes two main functional components: the traffic meter and the meter reader. The traffic meter follows a “rule-set” (packet filter) to identify the flow of packets to monitor. A packet-matching engine performs the packet classification to identify the flow to which a packet belongs according to the rules defined. The traffic meter measures specific attributes of the identified traffic flows, and records the measurements in a flow table. The meter reader will query the contents of the flow table for data analysis. The following are the benefits of using the RTFM architecture:                All data on the network is recordable—all data transmitted over a network can be tracked, recorded, and aggregated without affecting the data flow (assuming necessary implementation steps have been taken to respect node performance constraints that eventually relate to limited execution clock cycles).        Independence from physical and data layer technologies (i.e. analog phone line, fiber-optic) used to transmit the data between source and destination—these details are not required in the measurement of network flows. Flow monitoring gives a logical view of the network more suitable for QoS management than the physical view.        Good performance even during fault conditions—Flow meters are far less vulnerable than other parts of the network infrastructure since they passively intercept the data packets, thus maintaining accurate bandwidth consumption statistics where other parts of the network fail. Flow metering can be used even if the given network elements are incapable of flow monitoring.        Flows have a meaningful granularity—Flexible, comprehensive and scalable aggregation strategies can easily be provisioned.        
The approach contemplated by the invention is to make the core network operate and appear like a single router. This concept is discussed in a publication by Hakata et al. entitled “IP Core Transport Network”, Fujitsu Sci. Tech. J., 37 pp 12-21 (June 2001). Most of the intelligence is moved to the edge node, and the core consists of a simple, very high capacity data transport mechanism. Incoming IP packets are processed at the ingress node and sent to paths assigned between the ingress and egress nodes.
The internal transfer of IP packets in the virtual-router network is done based on a switching technology (layer 2 label switching: MPLS) which is less time consuming than conventional routing algorithm. This architecture relieves the IP data from the hop-by-hop processing required for the current router networks that execute IP layer processing in each core router/node. Basically, nodes except edge nodes do not need to perform IP layer processing and just need to handle layer 2 label switching (MPLS).
The best-effort path and QoS class paths are prepared at the edge node where each IP packet is properly assigned to the appropriate path. The QoS class path is provided by using the DiffServ capability of MPLS. This means that different QoS class paths can be realized without the need of too much packet processing power at the core node.
The interfaces (edge nodes) of the virtual-router network search for the optimal routes automatically, set explicit paths, and balance loads by splitting the IP flows into multiple paths. In this way, the virtual-router ensures a high utilization of the core network resources and improves the service quality by avoiding congestion.
One practical implementation of the aggregation and correlation aspects of the invention allows performing performance measurement on a per flow basis. When providers offer a service, they are tied to their customers by contracts that guarantee the quality of the services paid for. One of the characteristics of the traffic used to qualify the agreement is the delay, i.e. the end-to-end delivery time of packets. One of the usual terms of an SLA is that a customer agrees to pay a certain price for a connection service, but only if the traffic delay is less than a certain threshold. If the delay becomes longer, then the provider did not respect the contract which implies financial penalties. So obtaining reliable performance information on sensitive traffic for customers is of utmost importance for all parties to justify the commercial value of the service provided.
The importance of providing per customer traffic information has led to the development of per flow monitoring technologies, the most popular implementation being the aforementioned Netflow from Cisco.
However, none of the existing flow monitoring solutions gives a solution to perform per flow traffic performance measurement.
The best tools available to compute performance apply active measurement techniques based on PING wherein ICMP packets are sent from one point to another. These packets provide sequence numbering and timing information based on which packet loss and delay are computed. Evolutions have been brought to perform PING on specific quality of services (matching the CoS of customer traffic).
Data on customer usage of service can be provided by flow monitoring solution, and service performance can be computed by PING like tools. But nothing exists to correlate the information received from a PING to a specific customer flow. PING gives status on a link between two end points at a specific point in time, but do NOT provide the accurate performance experienced by a specific customer flow.
Furthermore a PING is an active measurement method. It works by introducing new traffic to the existing traffic. It thus impacts the overall traffic performance and still does not characterize real traffic.
Other intrusive solutions have been foreseen. For instance, by encapsulating the customer traffic in specific packet format with special processing at the egress side, or by tagging a packet. But these solutions are not yet satisfactory as customer packets are “touched”.